Combining Dynamic and Static Scheduling on Distributed-Memory Multiprocessors

Loops are a large source of parallelism for many numerical applications. An important issue in the parallel execution of loops is how to schedule them so that the work-load is well balanced among the processors. Most existing loop scheduling algorithms were designed for shared-memory multiprocessors, with uniform memory access costs. These approaches are not suitable for distributed-memory multipro-cessors where data locality is a major concern and communication costs are high. This paper presents a new scheduling algorithm in which data locality is taken into account. Our approach combines both worlds, static and dynamic scheduling , in a two-level (overlapped) fashion. This way data locality is considered and communication costs are limited. The performance of the new algorithm is evaluated on a CM-5 message-passing distributed-memory multiprocessor. 1 Introduction Loops exhibit most of the parallelism present in numerical programs. Therefore, distributing the workload of a loop evenly among the processors is a critical factor in the ef-cient execution of this kind of parallel programs. A loop scheduling strategy assigns iterations to the processors of the machine in such a way that all of them nish their work-load at more or less the same time. A simple strategy is static scheduling, which determines the assignment of iterations to processors at compile-time. But in many situations, the workload of the iterations is unpredictable at compile-time. Dynamic scheduling strategies have been developed to handle such situations, solving the assignment problem at run-time. When considering cross-iteration data dependencies we can classify the loops into three major types, doserial, doall and doacross Pol88]. In this paper we are primarily concerned with doall or parallel loops, in which there is no data dependence between any pair of their iterations. Otherwise,